Determining intra prediction mode of image coding unit and image decoding unit

ABSTRACT

A method and apparatus for determining an intra prediction mode of a coding unit. Candidate intra prediction modes of a chrominance component coding unit, which includes an intra prediction mode of a luminance component coding unit, are determined, and costs of the chrominance component coding unit according to the determined candidate intra prediction modes are compared to determine a minimum cost intra prediction mode to be the intra prediction mode of the chrominance component coding unit.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application is a continuation application of U.S. patentapplication Ser. No. 13/080,021, filed Apr. 5, 2011, which claimspriority from Korean Patent Application No. 10-2010-0031145, filed onApr. 5, 2010, in the Korean Intellectual Property Office, the disclosureof which is incorporated herein by reference in its entirety.

BACKGROUND

1. Field

Exemplary embodiments relate to encoding and decoding of an image, andmore particularly, to methods and apparatuses for encoding and decodingan image, wherein intra prediction is performed on a chrominancecomponent coding unit by applying an intra prediction mode determinedfor a luminance component coding unit.

2. Description of the Related Art

In an image compression method, such as Moving Picture Experts Group(MPEG)-1, MPEG-2, MPEG-4, or H.264/MPEG-4 Advanced Video Coding (AVC), apicture is divided into macroblocks to encode an image. Each of themacroblocks is encoded in all encoding modes that can be used in interprediction or intra prediction, and then is encoded in an encoding modethat is selected according to a bit rate used to encode the macroblockand a distortion degree of a decoded macroblock based on the originalmacroblock.

As hardware for reproducing and storing high resolution or high qualityvideo content is being developed and supplied, there is an increasingneed for a video codec capable of effectively encoding or decoding thehigh resolution or high quality video content. In a conventional videocodec, a video is encoded in units of macroblocks each having apredetermined size.

SUMMARY

The exemplary embodiments include a method of determining an intraprediction mode of a luminance component coding unit having variousdirectionality based on hierarchical coding units having various sizes,and methods and apparatuses for encoding and decoding an image, whereinintra prediction is performed on a chrominance component coding unitaccording to candidate intra prediction modes including an intraprediction mode determined for a luminance component coding unit.

According to an aspect of an exemplary embodiment, there is provided amethod of determining an intra prediction mode of a coding unit of acurrent picture, the method comprising: splitting a luminance componentof the current picture into at least one luminance component coding unitbased on a maximum coding unit that is a coding unit in which thecurrent picture is encoded having a maximum size, and a depth thatindicates hierarchical split information of the maximum coding unit;determining an intra prediction mode of the at least one luminancecomponent coding unit; comparing costs of applying to a chrominancecomponent unit candidate intra prediction modes of the chrominancecomponent coding unit and the intra prediction mode of the at least oneluminance component coding unit; and determining an intra predictionmode of the chrominance component coding unit from among the candidateprediction modes of the chrominance component unit and the determinedintra prediction mode of the at least one luminance component codingunit having a minimum cost, based on a result of the comparing.

According to an aspect of an exemplary embodiment, there is provided anapparatus for determining an intra prediction mode of a coding unit of acurrent picture, the apparatus comprising: a luminance intra predictorthat determines an intra prediction mode of a luminance component codingunit that is split from a maximum coding unit that is a coding unit inwhich the current picture is encoded having a maximum size, and a depththat indicates hierarchical split information of the maximum codingunit; and a chrominance intra predictor that compares costs of applyingto a chrominance component coding unit that is split from the maximumcoding unit candidate intra prediction modes of the chrominancecomponent coding unit and the intra prediction mode of the luminancecomponent coding unit, and determines the intra prediction mode of thechrominance component coding unit from among the candidate predictionmodes of the chrominance component unit and the intra prediction mode ofthe luminance component coding unit having a minimum cost, based on aresult of the comparing.

According to an aspect of an exemplary embodiment, there is providedclaim a method of determining an intra prediction mode of a decodingunit of a current picture, the method comprising: extracting a maximumcoding unit that is a coding unit in which the current picture isencoded having a maximum size, and a depth that indicates hierarchicalsplit information of the maximum coding unit, from a bitstream;splitting a luminance component and a chrominance component of thecurrent picture to be decoded into at least one luminance componentdecoding unit and at least one chrominance component decoding unit,respectively, based on the maximum coding unit and depth; extractingintra prediction mode information that indicates an intra predictionmode applied to the at least one luminance component decoding unit andthe at least one chrominance component decoding unit, from thebitstream; and performing intra prediction on the at least one luminancecomponent decoding unit and the at least one chrominance componentdecoding unit based on the extracted intra prediction mode informationto decode the at least one luminance component decoding unit and the atleast one chrominance component decoding unit.

According to an aspect of an exemplary embodiment, there is provided anapparatus for decoding an image, the apparatus comprising: an entropydecoder that extracts a maximum coding unit that is a coding unit inwhich the current picture is encoded having a maximum size, a depth thatindicates hierarchical split information of the maximum coding unit, andintra prediction mode information that indicates an intra predictionmode applied to a luminance component decoding unit and a chrominancecomponent decoding unit to be decoded, from a bitstream; and an intraprediction performer that performs intra prediction on the luminancecomponent decoding unit and the chrominance component decoding unit todecode the luminance component decoding unit and the chrominancecomponent decoding unit, according to the extracted intra predictionmode.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects will become more apparent by describing indetail exemplary embodiments thereof with reference to the attacheddrawings in which:

FIG. 1 is a block diagram of an apparatus for encoding a video,according to an exemplary embodiment;

FIG. 2 is a block diagram of an apparatus for decoding a video,according to an exemplary embodiment;

FIG. 3 is a diagram for describing a concept of coding units, accordingto an exemplary embodiment;

FIG. 4 is a block diagram of an image encoder based on coding units,according to an exemplary embodiment of the present invention;

FIG. 5 is a block diagram of an image decoder based on coding units,according to an exemplary embodiment;

FIG. 6 is a diagram illustrating deeper coding units according to depthsand prediction units, according to an exemplary embodiment;

FIG. 7 is a diagram for describing a relationship between a coding unitand a transform unit, according to an exemplary embodiment;

FIG. 8 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment;

FIG. 9 is a diagram of deeper coding units according to depths,according to an exemplary embodiment;

FIGS. 10A and 10B are diagrams for describing a relationship betweencoding units, prediction units, and transform units, according to anexemplary embodiment;

FIG. 11 is a table showing encoding information according to codingunits, according to an exemplary embodiment;

FIGS. 12A through 12C are diagrams of formats of a luminance componentimage and a chrominance component image, according to exemplaryembodiments;

FIG. 13 is a table showing a number of intra prediction modes accordingto sizes of luminance component coding units, according to an exemplaryembodiment;

FIGS. 14A through 14C are diagrams for explaining intra prediction modesapplied to a luminance component coding unit having a predeterminedsize, according to an exemplary embodiment;

FIG. 15 is a diagram for explaining intra prediction modes applied to aluminance component coding unit having a predetermined size, accordingto an exemplary embodiment;

FIG. 16A through 16C are a reference diagram for explaining intraprediction modes of a luminance component coding unit having variousdirectionalities, according to an exemplary embodiment;

FIG. 17 is a reference diagram for explaining a bi-linear mode accordingto an exemplary embodiment;

FIG. 18 is a diagram for explaining a process of generating a predictionvalue of an intra prediction mode of a current luminance componentcoding unit, according to an exemplary embodiment;

FIG. 19A through 19B are a reference diagram for explaining a mappingprocess of intra prediction modes between luminance component codingunits having different sizes, according to an exemplary embodiment;

FIG. 20 is a reference diagram for explaining a process of mapping anintra prediction mode of a neighboring luminance component coding unitto one of representative intra prediction modes, according to anexemplary embodiment;

FIG. 21 is a diagram for explaining candidate intra prediction modesapplied to a chrominance component coding unit, according to anexemplary embodiment;

FIG. 22 is a block diagram of an intra prediction apparatus, accordingto an exemplary embodiment;

FIG. 23 is a flowchart illustrating a method of determining an intraprediction mode of a coding unit, according to an exemplary embodiment;and

FIG. 24 is a flowchart illustrating a method of determining an intraprediction mode of a decoding unit, according to an exemplaryembodiment.

FIG. 25 is a diagram for explaining a relationship between a currentpixel and neighboring pixels located on an extended line having adirectivity of (dx, dy);

FIG. 26 is a diagram for explaining a change in a neighboring pixellocated on an extended line having a directivity of (dx, dy) accordingto a location of a current pixel, according to an exemplary embodiment;and

FIGS. 27 and 28 are diagrams for explaining a method of determining anintra prediction mode direction, according to exemplary embodiments.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, the exemplary embodiments will be described more fully withreference to the accompanying drawings.

Hereinafter, a ‘coding unit’ refers to an encoding data unit in whichthe image data is encoded at an encoder side, and an encoded data unitin which the encoded image data is decoded at a decoder side. Also, a‘coded depth’ refers to a depth at which a coding unit is encoded.

FIG. 1 is a block diagram of a video encoding apparatus 100, accordingto an exemplary embodiment.

The video encoding apparatus 100 includes a maximum coding unit splitter110, a coding unit determiner 120, an image data output unit 130, and anencoding information output unit 140.

The maximum coding unit splitter 110 may split a current picture basedon a maximum coding unit for the current picture of an image. If thecurrent picture is larger than the maximum coding unit, image data ofthe current picture may be split into the at least one maximum codingunit. The maximum coding unit according to an exemplary embodiment maybe a data unit having a size of 32×32, 64×64, 128×128, 256×256, etc.,wherein a shape of the data unit is a square having a width and lengthin squares of 2. The image data may be output to the coding unitdeterminer 120 according to the at least one maximum coding unit.

A coding unit according to an exemplary embodiment may be characterizedby a maximum size and a depth. The depth denotes a number of times thecoding unit is spatially split from the maximum coding unit, and as thedepth deepens, deeper encoding units according to depths may be splitfrom the maximum coding unit to a minimum coding unit. A depth of themaximum coding unit is an uppermost depth and a depth of the minimumcoding unit is a lowermost depth. Since a size of a coding unitcorresponding to each depth decreases as the depth of the maximum codingunit deepens, a coding unit corresponding to an upper depth may includea plurality of coding units corresponding to lower depths.

As described above, the image data of the current picture is split intothe maximum coding units according to a maximum size of the coding unit,and each of the maximum coding units may include deeper coding unitsthat are split according to depths. Since the maximum coding unitaccording to an exemplary embodiment is split according to depths, theimage data of a spatial domain included in the maximum coding unit maybe hierarchically classified according to depths.

A maximum depth and a maximum size of a coding unit, which limit thetotal number of times a height and a width of the maximum coding unitare hierarchically split, may be predetermined. Such a maximum codingunit and maximum depth may be set in a picture or a slice unit. In otherwords, different maximum coding units and different maximum depths maybe set for each picture or slice, and a size of a minimum coding unitincluded in the maximum coding unit may be set according to the maximumdepth. As such, by setting the maximum coding unit and the maximum depthaccording to pictures or slices, encoding efficiency may be improved byencoding an image of a flat region by using the maximum coding unit, andcompression efficiency of an image may be improved by encoding an imagehaving high complexity by using a coding unit having a smaller size thanthe maximum coding unit.

The coding unit determiner 120 determines different maximum depthsaccording to maximum coding units. The maximum depth may be determinedbased on a rate-distortion (R-D) cost calculation. The determinedmaximum depth is output to the encoding information output unit 140, andthe image data according to maximum coding units is output to the imagedata output unit 130.

The image data in the maximum coding unit is encoded based on the deepercoding units corresponding to at least one depth equal to or below themaximum depth, and results of encoding the image data are compared basedon each of the deeper coding units. A depth having the least encodingerror may be selected after comparing encoding errors of the deepercoding units. At least one coded depth may be selected for each maximumcoding unit.

The size of the maximum coding unit is split as a coding unit ishierarchically split according to depths, and as the number of codingunits increases. Also, even if coding units correspond to a same depthin one maximum coding unit, it is determined whether to split each ofthe coding units corresponding to the same depth to a lower depth bymeasuring an encoding error of the image data of each coding unit,separately. Accordingly, even when image data is included in one maximumcoding unit, the image data is split into regions according to thedepths and the encoding errors may differ according to regions in theone maximum coding unit, and thus the coded depths may differ accordingto regions in the image data. In other words, the maximum coding unitmay be split into coding units having different sizes according todifferent depths. Thus, one or more coded depths may be determined inone maximum coding unit, and the image data of the maximum coding unitmay be split according to coding units of at least one coded depth.

Also, the coding units having different sizes in the maximum coding unitmay be predicted or transformed based on data units having differentsizes. In other words, the video encoding apparatus 100 may perform aplurality of operations for encoding an image based on data units havingvarious sizes and shapes. In order to encode image data, operations suchas prediction, transformation, entropy encoding, etc. are performed, andat this time, the same data unit may be used for all operations ordifferent data units may be used for each operation.

For example, the video encoding apparatus 100 may select a data unitthat is different from the coding unit, to predict the coding unit. Forexample, when a coding unit has a size of 2N×2N (where N is a positiveinteger), a data unit for prediction may have a size of 2N×2N, 2N×N,N×2N, or N×N. In other words, motion prediction may be performed basedon a data unit obtained by splitting at least one of a height and awidth of the coding unit. Hereinafter, the data unit that is a basisunit of prediction will be referred to as a “prediction unit”.

A prediction mode may be at least one of an intra mode, an inter mode,and a skip mode, wherein a certain prediction mode is only performed ona prediction unit having a certain size or shape. For example, an intramode may be performed only on a square prediction unit having a size of2N×2N or N×N. Also, a skip mode may be performed only on a predictionunit having a size of 2N×2N. If a plurality of prediction units areincluded in the coding unit, prediction may be performed on eachprediction unit to select a prediction mode having a minimum error.

Alternatively, the video encoding apparatus 100 may transform the imagedata based on a data unit that is different from the coding unit. Inorder to transform the coding unit, transformation may be performedbased on a data unit having a size smaller than or equal to the codingunit. A data unit used as a base of the transformation will be referredto as a “transform unit”.

The coding unit determiner 120 may measure an encoding error of deepercoding units according to depths by using Rate-Distortion Optimizationbased on Lagrangian multipliers, to determine a spilt shape of themaximum coding unit having an optimum encoding error. In other words,the coding unit determiner 120 may determine shapes of the coding unitsto be split from the maximum coding unit, wherein the sizes of thecoding units are different according to depths.

The image data output unit 130 outputs the image data of the maximumcoding unit, which is encoded based on the at least one coded depthdetermined by the coding unit determiner 120, in bitstreams. Since theencoding is already performed by the coding depth determiner 120 tomeasure the minimum encoding error, an encoded data stream may be outputby using the minimum encoding error.

The encoding information output unit 140 may output information aboutthe encoding mode according to coded depth, which is encoded based onthe at least one coded depth determined by the coding unit determiner120, in bitstreams. The information about the encoding mode according tocoded depth may include information that indicates the coded depth,information that indicates split type in the prediction unit,information that indicates the prediction mode, and information thatindicates the size of the transform unit.

The information about the coded depth may be defined by using splitinformation according to depths, which indicates whether encoding isperformed on coding units of a lower depth instead of a current depth.If the current depth of the current coding unit is the coded depth,image data in the current coding unit is encoded and output, and thusthe split information may be defined not to split the current codingunit to a lower depth. Alternatively, if the current depth of thecurrent coding unit is not the coded depth, the encoding is performed onthe coding unit of the lower depth, and thus the split information maybe defined to split the current coding unit to obtain the coding unitsof the lower depth.

If the current depth is not the coded depth, encoding is performed onthe coding unit that is split into the coding unit of the lower depth.Since at least one coding unit of the lower depth exists in one codingunit of the current depth, the encoding is repeatedly performed on eachcoding unit of the lower depth, and thus the encoding may be recursivelyperformed for the coding units having the same depth.

Since the coding units having a tree structure are determined for onemaximum coding unit, and information about at least one encoding mode isdetermined for a coding unit of a coded depth, information about atleast one encoding mode may be determined for one maximum coding unit.Also, a coded depth of the image data of the maximum coding unit may bedifferent according to locations since the image data is hierarchicallysplit according to depths, and thus information about the coded depthand the encoding mode may be set for the image data.

Accordingly, the encoding information output unit 140 may assigncorresponding encoding information to each minimum coding unit includedin the maximum coding unit. In other words, the coding unit of the codeddepth includes at least one minimum coding unit containing the sameencoding information. Thus, if neighboring minimum coding units have thesame encoding information, the neighboring minimum coding units may bethe minimum coding units included in the same maximum coding unit.

In the video encoding apparatus 100, the deeper coding unit may be acoding unit obtained by dividing a height or width of a coding unit ofan upper depth, which is one layer above, by two. In other words, whenthe size of the coding unit of the current depth is 2N×2N, the size ofthe coding unit of the lower depth is N×N. Also, the coding unit of thecurrent depth having the size of 2N×2N may include a maximum of 4 of thecoding units of the lower depth.

Accordingly, the video encoding apparatus 100 may determine coding unitshaving an optimum shape for each maximum coding unit, based on the sizeof the maximum coding unit and the maximum depth determined consideringcharacteristics of the current picture. Also, since encoding may beperformed on each maximum coding unit by using any one of variousprediction modes and transforms, an optimum encoding mode may bedetermined considering characteristics of the coding unit of variousimage sizes.

Thus, if an image having high resolution or a large data amount isencoded in a conventional macroblock, a number of macroblocks perpicture excessively increases. Accordingly, a number of pieces ofcompressed information generated for each macroblock increases, and thusit is difficult to transmit the compressed information and datacompression efficiency decreases. However, by using the video encodingapparatus 100, image compression efficiency may be increased since acoding unit is adjusted while considering characteristics of an imagewhile increasing a maximum size of a coding unit while considering asize of the image.

FIG. 2 is a block diagram of a video decoding apparatus 200, accordingto an exemplary embodiment.

Referring to FIG. 2, the video decoding apparatus 200 includes areceiver 210, an encoding information extractor 220, and an image datadecoder 230.

The receiver 210 receives and parses a bitstream received by the videodecoding apparatus 200 to obtain image data according to maximum codingunits, and outputs the image data to the image data decoder 230. Thereceiver 210 may extract information about the maximum coding unit of acurrent picture or slice from a header about the current picture orslice. The video decoding apparatus 200 decodes the image data accordingto maximum coding units.

The encoding information extractor 220 parses a bitstream received bythe video decoding apparatus 200 and extracts information about a codeddepth and encoding mode according to maximum coding units from theheader of the current picture in the parsed bitstream. The informationabout the extracted coded depth and encoding mode are output to theimage data decoder 230.

The information about the coded depth and the encoding mode according tothe maximum coding unit may be set for information about at least onecoding unit corresponding to the coded depth, and information about anencoding mode may include split type information of a prediction unitaccording to coding units, information that indicates a prediction mode,and information that indicates a size of a transform unit. Also, splitinformation according to depths may be extracted as the informationabout the coded depth.

Information about a split shape of the maximum coding unit may includeinformation about coding units having different sizes according todepths, and information about an encoding mode may include informationthat indicates a prediction unit according to coding units, informationthat indicates a prediction mode, and information that indicates atransform unit.

The image data decoder 230 restores the current picture by decoding theimage data in each maximum coding unit based on the informationextracted by the encoding information extractor 220. The image datadecoder 230 may decode the coding unit included in the maximum codingunit based on the information about the split shape of the maximumcoding unit. A decoding process may include prediction, including intraprediction and motion compensation, and inverse transform.

Alternatively, the image data decoder 230 restores the current pictureby decoding the image data in each maximum coding unit based on theinformation about the coded depth and the encoding mode according to themaximum coding units. In other words, the image data decoder 230 maydecode the image data according to coding units of at least one codeddepth, based on the information about the coded depth according to themaximum coding units. A decoding process may include prediction,including intra prediction and motion compensation, and inversetransform.

The image data decoder 230 may perform intra prediction or motioncompensation in a prediction unit and a prediction mode according tocoding units, based on the information about the split type and theprediction mode of the prediction unit of the coding unit according tocoded depths, to perform prediction according to coding units. Also, theimage data decoder 230 may perform inverse transform according to eachtransform unit in the coding unit, based on the information about thesize of the transform unit of the coding unit according to coded depths,to perform the inverse transform according to maximum coding units.

The image data decoder 230 may determine a coded depth of a currentmaximum coding unit by using split information according to depth. Ifthe split information indicates decoding be performed at the currentdepth, the current depth is a coded depth. Accordingly, the image datadecoder 230 may decode encoded image data of a coding unit of thecurrent depth with respect to the image data of the current maximumcoding unit by using the information about the split type of theprediction unit, the prediction mode, and the size of the transformunit. In other words, the encoding information assigned to the minimumcoding unit may be observed, and the minimum coding units including theencoding information having the same split information may be gatheredto be decoded in one data unit.

The video decoding apparatus 200 may obtain information about at leastone coding unit that generates the minimum encoding error when encodingis recursively performed for each maximum coding unit, and may use theinformation to decode the current picture. In other words, the imagedata may be decoded in the optimum coding unit in each maximum codingunit. Accordingly, even if image data has high resolution and a largeamount of data, the image data may be efficiently decoded and restoredby using a size of a coding unit and an encoding mode, which areadaptively determined according to characteristics of the image data, byusing information about an optimum encoding mode received from anencoder.

FIG. 3 is a diagram for describing a concept of coding units accordingto an exemplary embodiment.

Referring to FIG. 3, a size of a coding unit may be expressed inwidth×height, and may be 64×64, 32×32, 16×16, 8×8 and 4×4. Aside fromthe coding unit having a square shape, the coding unit may have a sizeof 64×32, 32×64, 32×16, 16×32, 16×8, 8×16, 8×4, or 4×8.

In video data 310, a resolution is 1920×1080, a maximum size of a codingunit is 64, and a maximum depth is 2. In video data 320, a resolution is1920×1080, a maximum size of a coding unit is 64, and a maximum depth is4. In video data 330, a resolution is 352×288, a maximum size of acoding unit is 16, and a maximum depth is 2.

If a resolution is high or a data amount is large, a maximum size of acoding unit may be large to not only increase encoding efficiency butalso to accurately reflect characteristics of an image. Accordingly, themaximum size of the coding unit of the video data 310 and 320 having ahigher resolution than the video data 330 may be 64.

The maximum depth denotes a total number of splits from a maximum codingunit to a minimum decoding unit. Accordingly, since the maximum depth ofthe video data 310 is 2, coding units 315 of the video data 310 mayinclude a maximum coding unit having a long axis size of 64, and codingunits having long axis sizes of 32 and 16 since depths are deepened bytwo layers by splitting the maximum coding unit twice. Meanwhile, sincethe maximum depth of the video data 330 is 2, coding units 335 of thevideo data 330 may include a maximum coding unit having a long axis sizeof 16, and coding units having long axis size of 8 and 4 since depthsare deepened by two layers by splitting the maximum coding unit twice.

Since the maximum depth of the video data 320 is 4, coding units 325 ofthe video data 320 may include a maximum coding unit having a long axissize of 64, and coding units having long axis sizes of 32, 16, 8, and 4since the depths are deepened by 4 layers by splitting the maximumcoding unit four times. As a depth deepens, detailed information may beprecisely expressed.

FIG. 4 is a block diagram of an image encoder 400 based on coding units,according to an exemplary embodiment.

Referring to FIG. 4, an intra predictor 410 performs intra prediction oncoding units in an intra mode, from among coding units of a currentframe 405, and a motion estimator 420 and a motion compensator 425performs inter estimation and motion compensation on coding units in aninter mode from among coding units of the current frame 405 by using thecurrent frame 405, and a reference frame 495.

Data output from the intra predictor 410, the motion estimator 420, andthe motion compensator 425 is output as a quantized transformcoefficient through a transformer 430 and a quantizer 440. The quantizedtransform coefficient is restored as data in a spatial domain through aninverse quantizer 460 and an inverse transformer 470, and the restoreddata in the spatial domain is output as the reference frame 495 afterbeing post-processed through a deblocking unit 480 and a loop filteringunit 490. The quantized transform coefficient may be output as abitstream 455 through an entropy encoder 450.

In order for the image encoder 400 to be applied in the video encodingapparatus 100, all elements of the image encoder 400, i.e., the intrapredictor 410, the motion estimator 420, the motion compensator 425, thetransformer 430, the quantizer 440, the entropy encoder 450, the inversequantizer 460, the inverse transformer 470, the deblocking unit 480, andthe loop filtering unit 490 perform image encoding processes based onthe maximum coding unit, the coding unit according to depths, theprediction unit, and the transform unit. Specifically, the intrapredictor 410, the motion estimator 420, and the motion compensator 425determine a prediction unit and a prediction mode of a coding unit byconsidering a maximum size and depth of the coding unit, and thetransformer 430 determines the size of the transform unit by consideringthe maximum size and depth of the coding unit. Also, as described later,the intra predictor 410 performs intra prediction by applying an intraprediction mode determined for a luminance component coding unit on achrominance component coding unit, and thus prediction efficiency of thechrominance component coding unit may be improved.

FIG. 5 is a block diagram of an image decoder 500 based on coding units,according to an exemplary embodiment.

Referring to FIG. 5, a parser 510 parses a received bitstream 505 andextracts encoded image data to be decoded and information about encodingrequired for decoding from the parsed bitstream. The encoded image datais output as inverse quantized data through an entropy decoder 520 andan inverse quantizer 530, and the inverse quantized data is restored toimage data in a spatial domain through an inverse transformer 540. Anintra predictor 550 performs intra prediction on coding units in anintra mode with respect to the image data in the spatial domain, and amotion compensator 560 performs motion compensation on coding units inan inter mode by using a reference frame 585. The image data in thespatial domain, which passed through the intra predictor 550 and themotion compensator 560, may be output as a restored frame 595 afterbeing post-processed through a deblocking unit 570 and a loop filteringunit 580. Also, the image data that is post-processed through thedeblocking unit 570 and the loop filtering unit 580 may be output as thereference frame 585.

In order for the image decoder 500 to be applied in the video decodingapparatus 200, all elements of the image decoder 500, i.e., the parser510, the entropy decoder 520, the inverse quantizer 530, the inversetransformer 540, the intra predictor 550, the motion compensator 560,the deblocking unit 570, and the loop filtering unit 580 perform imagedecoding processes based on the maximum coding unit, the coding unitaccording to depths, the prediction unit, and the transform unit.Specifically, the intra prediction 550 and the motion compensator 560determine the prediction unit and the prediction mode of the coding unitby considering the maximum size and depth of the coding unit, and theinverse transformer 540 determines the size of transform unit byconsidering the maximum size and depth of the coding unit.

FIG. 6 is a diagram illustrating deeper coding units according todepths, and prediction units according to an exemplary embodiment.

The video encoding apparatus 100 and the video decoding apparatus 200use hierarchical coding units to consider characteristics of an image. Amaximum height, a maximum width, and a maximum depth of coding units maybe adaptively determined according to the characteristics of the image,or may be individually set according to an input of a user. Sizes ofdeeper coding units according to depths may be determined according tothe predetermined maximum size of the coding unit.

In a hierarchical structure 600 of coding units, according to anexemplary embodiment, the maximum height and the maximum width of thecoding units are each 64, and the maximum depth is 4. Since a depthdeepens along a vertical axis of the hierarchical structure 600, aheight and a width of the deeper coding unit are each split. Also, aprediction unit constituting a partial data unit, which is a base forprediction encoding of each deeper coding unit, is shown along ahorizontal axis of the hierarchical structure 600.

In other words, a coding unit 610 is a maximum coding unit in thehierarchical structure 600, wherein a depth is 0 and a size, i.e., aheight by width, is 64×64. The depth deepens along the vertical axis,and a coding unit 620 having a size of 32×32 and a depth of 1, a codingunit 630 having a size of 16×16 and a depth of 2, a coding unit 640having a size of 8×8 and a depth of 3, and a coding unit 650 having asize of 4×4 and a depth of 4 exist. The coding unit 650 having the sizeof 4×4 and the depth of 4 is a minimum coding unit.

Partial data units are shown in FIG. 6 as the prediction units of acoding unit along the horizontal axis according to each depth. In otherwords, if the coding unit 610 having the size of 64×64 and the depth of0 is a prediction unit, the prediction unit may be split into partialdata units included in the encoding unit 610, i.e. a partial data unit610 having a size of 64×64, partial data units 612 having the size of64×32, partial data units 614 having the size of 32×64, or partial dataunits 616 having the size of 32×32.

A prediction unit of the coding unit 620 having the size of 32×32 andthe depth of 1 may be split into partial data units included in thecoding unit 620, i.e. a partial data unit 620 having a size of 32×32,partial data units 622 having a size of 32×16, partial data units 624having a size of 16×32, and partial data units 626 having a size of16×16.

A prediction unit of the coding unit 630 having the size of 16×16 andthe depth of 2 may be split into partial data units included in thecoding unit 630, i.e. a partial data unit having a size of 16×16included in the coding unit 630, partial data units 632 having a size of16×8, partial data units 634 having a size of 8×16, and partial dataunits 636 having a size of 8×8.

A prediction unit of the coding unit 640 having the size of 8×8 and thedepth of 3 may be split into partial data units included in the codingunit 640, i.e. a partial data unit having a size of 8×8 included in thecoding unit 640, partial data units 642 having a size of 8×4, partialdata units 644 having a size of 4×8, and partial data units 646 having asize of 4×4.

The coding unit 650 having the size of 4×4 and the depth of 4 is theminimum coding unit and a coding unit of the lowermost depth. Aprediction unit of the coding unit 650 is only assigned to a partialdata unit having a size of 4×4.

In order to determine the at least one coded depth of the coding unitsconstituting the maximum coding unit 610, the coding unit determiner 120of the video encoding apparatus 100 performs encoding for coding unitscorresponding to each depth included in the maximum coding unit 610.

A number of deeper coding units according to depths including data inthe same range and the same size increases as the depth deepens. Forexample, four coding units corresponding to a depth of 2 are required tocover data that is included in one coding unit corresponding to a depthof 1. Accordingly, in order to compare encoding results of the same dataaccording to depths, the coding unit corresponding to the depth of 1 andfour coding units corresponding to the depth of 2 are each encoded.

In order to perform encoding for a current depth from among the depths,a minimum encoding error may be determined for the current depth byperforming encoding for each prediction unit in the coding unitscorresponding to the current depth, along the horizontal axis of thehierarchical structure 600. Also, a minimum encoding error according todepths may be searched for by comparing the minimum encoding error ofeach depth, by performing encoding for each depth as the depth deepensalong the vertical axis of the hierarchical structure 600. A depthhaving the minimum encoding error in the coding unit 610 may be selectedas the coded depth and a split type of the coding unit 610.

FIG. 7 is a diagram for describing a relationship between a coding unit710 and transform units 720, according to an exemplary embodiment.

The video encoding apparatus 100 or 200 encodes or decodes an imageaccording to coding units having sizes smaller than or equal to amaximum coding unit for each maximum coding unit. Sizes of transformunits for transform during encoding may be selected based on data unitsthat are not larger than a corresponding coding unit. For example, inthe video encoding apparatus 100 or 200, if a size of the coding unit710 is 64×64, transform may be performed by using the transform units720 having a size of 32×32. Also, data of the coding unit 710 having thesize of 64×64 may be encoded by performing the transform on each of thetransform units having the size of 32×32, 16×16, 8×8, and 4×4, which aresmaller than 64×64, and then a transform unit having the least codingerror may be selected.

FIG. 8 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment.

The output unit 130 of the video encoding apparatus 100 may encode andtransmit information 800 that indicates a split type, information 810that indicates a prediction mode, and information 820 that indicates asize of a transform unit for each coding unit corresponding to a codeddepth, as information about an encoding mode.

The information 800 includes information about a split type of aprediction unit of a current coding unit, wherein a split predictionunit is a data unit for prediction encoding the current coding unit. Forexample, a current coding unit CU_(—)0 having a depth 0 and size of2N×2N may be split into any one of a prediction unit 802 having a sizeof 2N×2N, a prediction unit 804 having a size of 2N×N, a prediction unit806 having a size of N×2N, and a prediction unit 808 having a size ofN×N. Here, the information 800 about a split type is set to indicate oneof the prediction unit 804 having a size of 2N×N, the prediction unit806 having a size of N×2N, and the prediction unit 808 having a size ofN×N

The information 810 indicates a prediction mode of each prediction unit.For example, the information 810 may indicate a mode of predictionencoding performed on a prediction unit indicated by the information800, i.e., an intra mode 812, an inter mode 814, or a skip mode 816.

The information 820 indicates a transform unit to be based on whentransform is performed on a current coding unit. For example, thetransform unit may be a first intra transform unit 822, a second intratransform unit 824, a first inter transform unit 826, or a second intratransform unit 828.

The encoding information extractor 220 of the video decoding apparatus200 may extract and use the information 800, 810, and 820 for decoding,according to each deeper coding unit FIG. 9 is a diagram of deepercoding units according to depths, according to an exemplary embodiment.

Split information may be used to indicate a change in depth. The spiltinformation indicates whether a coding unit of a current depth is splitinto coding units of a lower depth.

A prediction unit 910 for prediction encoding a coding unit having adepth of 0 and a size of 2N_(—)0×2N_(—)0 may include a split type 912having a size of 2N_(—)0×2N_(—)0, a split type 914 having a size of2N_(—)0×N_(—)0, a split type 916 having a size of N_(—)0×2N_(—)0, and asplit type 918 having a size of N_(—)0×N_(—)0.

Encoding via motion prediction is repeatedly performed on one predictionunit having a size of 2N_(—)0×2N_(—)0, two prediction units having asize of 2N_(—)0×N_(—)0, two prediction units having a size ofN_(—)0×2N_(—)0, and four prediction units having a size ofN_(—)0×N_(—)0, according to each split type. The prediction in an intramode and an inter mode may be performed on the prediction units havingthe sizes of 2N_(—)0×N_(—)0, N_(—)0×2N_(—)0 and N_(—)0×N_(—)0 andN_(—)0×N_(—)0. The motion prediction in a skip mode is performed only onthe prediction unit having the size of 2N_(—)0×2N_(—)0.

If the encoding error is the smallest in the split type 918 having thesize N_(—)0×N_(—)0, a depth is changed from 0 to 1 to split thepartition type 918 in operation 920, and encoding is repeatedlyperformed on coding units 922, 924, 926, and 928 having a depth of 2 anda size of N_(—)0×N_(—)0 to search for a minimum encoding error.

Since the encoding is repeatedly performed on the coding units 922, 924,926, and 928 having the same depth, only encoding of a coding unithaving a depth of 1 will be described as an example. A prediction unit930 for motion predicting a coding unit having a depth of 1 and a sizeof 2N_(—)1×2N_(—)1 (=N_(—)0×N_(—)0) may include a split type 932 havinga size of 2N_(—)1×2N_(—)1, a split type 934 having a size of2N_(—)1×N_(—)1, a split type 936 having a size of N_(—)1×2N_(—)1, and asplit type 938 having a size of N_(—)1×N_(—)1. Encoding via motionprediction is repeatedly performed on one prediction unit having a sizeof 2N_(—)1×2N_(—)1, two prediction units having a size of2N_(—)1×N_(—)1, two prediction units having a size of N_(—)1×2N_(—)1,and four prediction units having a size of N_(—)1×N_(—)1, according toeach split type.

If an encoding error is the smallest in the split type 938 having thesize of N_(—)1×N_(—)1, a depth is changed from 1 to 2 to split the splittype 938 in operation 940, and encoding is repeatedly performed oncoding units 942, 944, 946, and 948, which have a depth of 2 and a sizeof N_(—)2×N_(—)2 to search for a minimum encoding error.

When a maximum depth is d, split information according to each depth maybe set up to when a depth becomes d−1. In other words, a prediction unit950 for motion predicting a coding unit having a depth of d−1 and a sizeof 2N_(d−1)×2N_(d−1) may include a split type 952 having a size of2N_(d−1)×2N_(d−1), a split type 954 having a size of 2N_(d−1)×N_(d−1), asplit type 956 having a size of N_(d−1)×2N_(d−1), and a split type 958having a size of N_(d−1)×N_(d−1).

Encoding via motion prediction may be repeatedly performed on oneprediction unit having a size of 2N_(d−1)×2N_(d−1), two prediction unitshaving a size of 2N_(d−1)×N_(d−1), two prediction units having a size ofN_(d−1)×2N_(d−1), and four prediction units having a size ofN_(d−1)×N_(d−1), according to each split type. Since the maximum depthis d, a coding unit 952 having a depth of d−1 is not split.

In order to determine a coded depth for the coding unit 912, the videoencoding apparatus 100 selects a depth having the least encoding errorby comparing encoding errors according to depths. For example, anencoding error of a coding unit having a depth of 0 may be encoded byperforming motion prediction on each of the split types 912, 914, 916,and 918, and than a prediction unit having the least encoding error maybe determined. Similarly, a prediction unit having the least encodingerror may be searched for, according to depths 0 through d−1. In a depthof d, an encoding error may be determined by performing motionprediction on the prediction unit 960 having the size of 2N_d×2N_d. Assuch, the minimum encoding errors according to depths are compared inall of the depths of 1 through d, and a depth having the least encodingerror may be determined as a coded depth. The coded depth and theprediction unit of the corresponding coded depth mode may be encoded andtransmitted as information about an encoding mode. Also, since a codingunit is split from a depth of 0 to a coded depth, only split informationof the coded depth is set to 0, and split information of depthsexcluding the coded depth is set to 1.

The image data and encoding information extractor 220 of the videodecoding apparatus 200 may extract and use the information about thecoded depth and the prediction unit of the coding unit 912 to decode thecoding unit 912. The video decoding apparatus 200 may determine a depth,in which split information is 0, as a coded depth by using splitinformation according to depths, and use information about an encodingmode of the corresponding depth for decoding.

FIGS. 10A and 10B are diagrams for describing a relationship betweencoding units 1010, prediction units 1060, and transform units 1070,according to an exemplary embodiment.

The coding units 1010 are coding units corresponding to coded depthsdetermined by the video encoding apparatus 100, in a maximum coding unit1000. The prediction units 1060 are prediction units of each of thecoding units 1010, and the transform units 1070 are transform units ofeach of the coding units 1010.

When a depth of a maximum coding unit is 0 in the coding units 1010,depths of coding units 1012 and 1054 are 1, depths of coding units 1014,1016, 1018, 1028, 1050, and 1052 are 2, depths of coding units 1020,1022, 1024, 1026, 1030, 1032, and 1048 are 3, and depths of coding units1040, 1042, 1044, and 1046 are 4.

In the prediction units 1060, some encoding units 1014, 1016, 1022,1032, 1048, 1050, 1052, and 1054 are obtained by splitting the codingunits in the encoding units 1010. In other words, split types in thecoding units 1014, 1022, 1050, and 1054 have a size of 2N×N, split typesin the coding units 1016, 1048, and 1052 have a size of N×2N, and asplit type of the coding unit 1032 has a size of N×N. Prediction unitsof the coding units 1010 are smaller than or equal to each coding unit.

Transform or inverse transform is performed on image data of the codingunit 1052 in the transform units 1070 in a data unit that is smallerthan the coding unit 1052. Also, the coding units 1014, 1016, 1022,1032, 1048, 1050, and 1052 in the transform units 1070 are differentfrom those in the prediction units 1060 in terms of sizes and shapes. Inother words, the video encoding and decoding apparatuses 100 and 200 mayperform intra prediction, motion estimation, motion compensation,transform, and inverse transform individually on a data unit in the samecoding unit.

FIG. 11 is a table showing encoding information according to codingunits, according to an exemplary embodiment.

The encoding information output unit 140 of the video encoding apparatus100 may encode the encoding information according to coding units, andthe encoding information extractor 220 of the video encoding apparatus200 may extract the encoding information according to coding units.

Encoding information may include split information about a coding unit,split type information, prediction mode information, and informationabout a size of a transform unit. The encoding information shown in FIG.11 is merely exemplary of information that may be set by the videoencoding apparatus 100 and the video decoding apparatus 200, and is notlimited thereto.

The split information may indicate a coded depth of a correspondingcoding unit. In other words, since a coded depth is a depth that is nolonger split according to the split information, the information aboutsplit type, prediction mode, and size of transform unit may be set forthe coded depth. If the current coding unit is further split accordingto the split information, encoding is independently performed on foursplit coding units of a lower depth.

The information about a split type may indicate a split type of atransform unit of a coding unit in a coded depth as one of 2N×2N, 2N×N,N×2N, and N×N. The prediction mode may indicate a motion prediction modeas one of an intra mode, an inter mode, and a skip mode. The intra modemay be defined only in the split types of 2N×2N and N×N, and the skipmode may be only defined in the split type of 2N×2N. The transform unitmay have two sizes in the intra mode, and two sizes in the inter mode.

The encoding information according to coding units in the coded depthmay be included in the minimum coding unit in the coding unit.Accordingly, by checking the encoding information included in theneighboring minimum coding units, it may be determined whether theneighboring minimum coding units are included in the coding units havingthe same coded depth. Also, since the coding unit of the correspondingcoded depth may be determined by using the encoding information includedin the minimum coding unit, distribution of the coded depths of theminimum coding units may be inferred.

Intra prediction performed by the intra prediction unit 410 of the videoencoding apparatus 100 illustrated in FIG. 4 and the intra predictionunit 550 of the video decoding apparatus 200 illustrated in FIG. 5 willnow be described in detail. In the following description, a coding unitrefers to a current encoded block in an encoding process of an image,and a decoding unit refers to a current decoded block in a decodingprocess of an image. The coding unit and the decoding unit are differentonly in that the coding unit is used in the encoding process and thedecoding unit is used in the decoding process. For consistency, exceptfor a particular case, the coding unit and the decoding unit arereferred to as a coding unit in both the encoding and decodingprocesses.

FIGS. 12A through 12C are diagrams of formats of a luminance componentimage and a chrominance component image, according to an exemplaryembodiment.

Each coding unit forming one frame may be expressed by using one ofthree components, i.e., Y, Cb, and Cr. Y is luminance data havingluminance information, and Cb and Cr are chrominance data havingchrominance information.

The chrominance data may be expressed using a lower amount of data thanthe luminance data, based on the premise that a person is generally moresensitive to the luminance information than the chrominance information.Referring to FIG. 12A, one coding unit having a 4:2:0 format includesluminance data 1210 having a size of H×W (H and W are positiveintegers), and two pieces of chrominance data 1220 and 1230 having asize of (H/2)×(W/2) obtained by sampling the chrominance components Cband Cr by 1/4. Referring to FIG. 12B, one coding unit having a 4:2:2format includes luminance data 1240 having a size of H×W (H and W arepositive integers), and two pieces of chrominance data 1250 and 1260having a size of H×(W/2) obtained by sampling the chrominance componentsCb and Cr by 1/2 in a horizontal direction. Also, referring to FIG. 12C,when one coding unit has a 4:4:4 format, the coding unit includesluminance data 1270, and chrominance data 1280 and 1290, each having asize of H×W without sampling the chrominance components Cb and Cr, toprecisely express a chrominance component image.

Hereinafter, it is assumed that the luminance component coding unit andthe chrominance component coding unit, which are intra predicted, areone of image signals having color formats of 4:2:0, 4:2:2, and 4:4:4defined in a YCbCr (or YUV) color domain.

Prediction efficiency of the chrominance coding unit is improved byincluding an intra prediction mode determined for the luminancecomponent coding unit in candidate intra prediction modes applied to thechrominance component coding unit by considering a relationship betweenthe luminance component and the chrominance component.

FIG. 13 is a table showing a number of intra prediction modes accordingto sizes of luminance component coding units, according to an exemplaryembodiment.

According to an exemplary embodiment, the number of intra predictionmodes to be applied to a luminance component coding unit (a decodingunit in a decoding process) may be variously set. For example, referringto FIG. 13, if the size of a luminance component coding unit is N×N, onwhich intra prediction is performed, the numbers of intra predictionmodes actually performed on 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, and128×128-sized luminance component coding units may be respectively setas 5, 9, 9, 17, 33, 5, and 5 (in Example 2). For another example, when asize of a luminance component coding unit to be intra-predicted is N×N,numbers of intra prediction modes to be actually performed on codingunits having sizes of 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, and 128×128may be set to be 3, 17, 34, 34, 34, 5, and 5. The numbers of intraprediction modes to be actually performed are differently set accordingto the sizes of luminance component coding units because overheads forencoding prediction mode information differ according to the sizes ofthe luminance component coding units. In other words, a small luminancecomponent coding unit occupies a small portion of entire image data butmay have a large overhead in order to transmit additional information,such as prediction mode information of the luminance component codingunit. Accordingly, if a luminance component small coding unit is encodedby using an excessively large number of prediction modes, the number ofbits may be increased and thus compression efficiency may be reduced.Also, a luminance component large coding unit, e.g., a luminancecomponent coding unit equal to or greater than 64×64, generallycorresponds to a plain region of image data, and thus encoding of thelarge luminance component coding unit by using an excessively largenumber of prediction modes may also reduce compression efficiency.

Thus, according to an exemplary embodiment, luminance component codingunits are roughly classified into at least three sizes such as N1×N1(where 2≦N1≦4, and N1 is an integer), N2×N2 (where 8≦N2≦32, and N2 is aninteger), and N3×N3 (where 64≦N3, and N3 is an integer). If the numberof intra prediction modes performed on the luminance component codingunits of N1×N1 is A1 (where A1 is a positive integer), the number ofintra prediction modes performed on the luminance component coding unitsof N2×N2 is A2 (where A2 is a positive integer), and the number of intraprediction modes performed on the luminance component coding units ofN3×N3 is A3 (where A3 is a positive integer). The numbers of intraprediction modes performed according to the sizes of the luminancecomponent coding units may be set to satisfy A3≦A1≦A2. That is, if acurrent picture is split into small luminance component coding units,medium luminance component coding units, and large luminance componentcoding units, the medium luminance component coding units may be set tohave the largest number of prediction modes and the small luminancecomponent coding units and the large luminance component coding unitsmay be set to have a relatively small number of prediction modes.However, the exemplary embodiment is not limited thereto, and the smalland large luminance component coding units may also be set to have alarge number of prediction modes. The numbers of prediction modesaccording to the sizes of luminance component coding units in FIG. 13are merely exemplarily and may be changed.

FIG. 14A is a table showing intra prediction modes applied to aluminance component coding unit having a predetermined size, accordingto an exemplary embodiment.

Referring to FIGS. 13 and 14A, for example, when intra prediction isperformed on a luminance component coding unit having a 4×4 size, theluminance component coding unit may have a vertical mode (mode 0), ahorizontal mode (mode 1), a direct current (DC) mode (mode 2), adiagonal down-left mode (mode 3), a diagonal down-right mode (mode 4), avertical-right mode (mode 5), a horizontal-down mode (mode 6), avertical-left mode (mode 7), and a horizontal-up mode (mode 8).

FIG. 14B illustrates directions of the intra prediction modes shown inFIG. 14A. In FIG. 14B, numbers at ends of arrows represent predictionmodes corresponding to prediction directions indicated by the arrows.Here, mode 2 is a DC mode having no directionality and thus is not shownin FIG. 16B.

FIG. 14C is a diagram for describing a method of performing intraprediction on a luminance component coding unit by using the intraprediction modes shown in FIG. 14A, according to an exemplaryembodiment.

Referring to FIG. 14C, a prediction coding unit is generated accordingto an available intra prediction mode determined according to the sizeof a current luminance component coding unit by using neighboring pixelsA through M of the current luminance component coding unit. For example,an operation of performing prediction encoding on a current coding unithaving a 4×4 size according to mode 0, i.e., a vertical mode, shown inFIG. 14A will be described. Initially, values of the neighboring pixelsA through D at an upper side of the current coding unit are predicted aspixel values of the current coding unit. That is, the value of theneighboring pixel A is predicted as a value of four pixels in a firstcolumn of the current coding unit, the value of the neighboring pixel Bis predicted as a value of four pixels in a second column of the currentcoding unit, the value of the neighboring pixel C is predicted as avalue of four pixels in a third column of the current coding unit, andthe value of the neighboring pixel D is predicted as a value of fourpixels in a fourth column of the current coding unit. After that, thepixel values of the current coding unit predicted by using theneighboring pixels A through D are subtracted from the pixel values ofthe original current coding unit to calculate an error value and thenthe error value is encoded.

FIG. 15 is a diagram for explaining intra prediction modes applied to aluminance component coding unit having a predetermined size, accordingto an exemplary embodiment.

Referring to FIGS. 13 and 15, for example, when intra prediction isperformed on a coding unit having a 2×2 size, the coding unit may have atotal of five modes, such as a vertical mode, a horizontal mode, a DCmode, a plane mode, and a diagonal down-right mode.

Meanwhile, if a luminance component coding unit having a 32×32 size has33 intra prediction modes, as shown in FIG. 13, directions of the 33intra prediction modes need to be set. According to an exemplaryembodiment, in order to set intra prediction modes having variousdirections in addition to the intra prediction modes illustrated inFIGS. 14 and 15, prediction directions for selecting neighboring pixelsused as reference pixels of pixels of the luminance component codingunit are set by using (dx, dy) parameters. For example, if each of the33 prediction modes is defined as mode N (where N is an integer from 0to 32), mode 0 may be set as a vertical mode, mode 1 may be set as ahorizontal mode, mode 2 may be set as a DC mode, mode 3 may be set as aplane mode, and each of mode 4 through mode 31 may be defined as aprediction mode having a directionality of tan⁻¹(dy/dx) by using (dx,dy) represented as one of (1,−1), (1,1), (1,2), (2,1), (1,−2), (2,1),(1,−2), (2,−1), (2,−11), (5,−7), (10,−7), (11,3), (4,3), (1,11), (1,−1),(12,−3), (1,−11), (1,−7), (3,−10), (5,−6), (7,−6), (7,−4), (11,1),(6,1), (8,3), (5,3), (5,7), (2,7), (5,−7), and (4,−3) as shown in Table1.

TABLE 1 mode # dx dy mode 4 1 −1 mode 5 1 1 mode 6 1 2 mode 7 2 1 mode 81 −2 mode 9 2 −1 mode 10 2 −11 mode 11 5 −7 mode 12 10 −7 mode 13 11 3mode 14 4 3 mode 15 1 11 mode 16 1 −1 mode 17 12 −3 mode 18 1 −11 mode19 1 −7 mode 20 3 −10 mode 21 5 −6 mode 22 7 −6 mode 23 7 −4 mode 24 111 mode 25 6 1 mode 26 8 3 mode 27 5 3 mode 28 5 7 mode 29 2 7 mode 30 5−7 mode 31 4 −3 Mode 0 is a vertical mode, mode 1 is a horizontal mode,mode 2 is a DC mode, mode 3 is a plane mode, and mode 32 is a bi-linearmode.

Mode 32 may be set as a bi-linear mode that uses bi-linear interpolationas will be described later with reference to FIG. 16.

FIG. 16A thorough 16C is a reference diagrams for explaining intraprediction modes of a luminance component coding unit having variousdirectionalities, according to an exemplary embodiment.

As described above with reference to Table 1, each of the intraprediction modes according to exemplary embodiments may havedirectionality of tan⁻¹(dy/dx) by using a plurality of (dx, dy)parameters.

Referring to FIG. 16A, neighboring pixels A and B on a line 160 thatextends from a current pixel P in a current luminance component codingunit, which is to be predicted, at an angle of tan⁻¹(dy/dx) determinedby a value of a (dx, dy) parameter according to a mode, as shown inTable 1, may be used as predictors of the current pixel P. In this case,the neighboring pixels A and B may be pixels that have been encoded andrestored, and belong to previous coding units located above and to theleft side of the current coding unit. Also, when the line 160 does notpass along neighboring pixels on locations each having an integral valuebut passes between these neighboring pixels, neighboring pixels closerto the line 160 may be used as predictors of the current pixel P. Also,a weighted average value considering a distance between an intersectionof the line 160 and neighboring pixels close to the line 160 may be usedas a predictor for the current pixel P. If two pixels that meet the line160, e.g., the neighboring pixel A located above the current pixel P andthe neighboring pixel B located to the left side of the current pixel P,are present, an average of pixel values of the neighboring pixels A andB may be used as a predictor of the current pixel P. Otherwise, if aproduct of values of the ‘dx’ and ‘dy’ parameters is a positive value,the neighboring pixel A may be used, and if the product of the values ofthe ‘dx’ and ‘dy’ parameters is a negative value, the neighboring pixelB may be used.

FIGS. 16B and 16C are reference diagrams for explaining a process ofgenerating a predictor when the line 160 of FIG. 16A passes between, notthrough, neighboring pixels of integer locations.

Referring to FIG. 16B, if the line 160 having an angle of tan⁻¹(dy/dx)that is determined according to (dx, dy) of each mode passes between aneighboring pixel A 161 and a neighboring pixel B 162 of integerlocations, a weighted average value considering a distance between anintersection of the extended line 160 and the neighboring pixels A 161and B 162 close to the extended line 160 may be used as a predictor asdescribed above. For example, if a distance between the neighboringpixel A 161 and the intersection of the extended line 160 having theangle of tan⁻¹(dy/dx) is f, and a distance between the neighboring pixelB 162 and the intersection of the extended line 160 is g, a predictorfor the current pixel P may be obtained as (A*g+B*f)/(f+g). Here, f andg may be each a normalized distance using an integer. If software orhardware is used, the predictor for the current pixel P may be obtainedby shift operation as (g*A+f*B+2)>>2. As shown in FIG. 16B, if theextended line 160 passes through a first quarter close to theneighboring pixel A 161 from among four parts obtained by quartering adistance between the neighboring pixel A 161 and the neighboring pixel B162 of the integer locations, the predictor for the current pixel P maybe acquired as (3*A+B)/4. Such operation may be performed by shiftoperation considering rounding-off to a nearest integer like(3*A+B+2)>>2.

Meanwhile, if the extended line 160 having the angle of tan⁻¹(dy/dx)that is determined according to (dx, dy) of each mode passes between theneighboring pixel A 161 and the neighboring pixel B 162 of the integerlocations, a section between the neighboring pixel A 161 and theneighboring pixel B 162 may be divided into a predetermined number ofareas, and a weighted average value considering a distance between anintersection and the neighboring pixel A 161 and the neighboring pixel B162 in each divided area may be used as a prediction value. For example,referring to FIG. 16C, a section between the neighboring pixel A 161 andthe neighboring pixel B 162 may be divided into five sections P1 throughP5 as shown in FIG. 16C, a representative weighted average valueconsidering a distance between an intersection and the neighboring pixelA 161 and the neighboring pixel B 162 in each section may be determined,and the representative weighted average value may be used as a predictorfor the current pixel P. In detail, if the extended line 160 passesthrough the section P1, a value of the neighboring pixel A may bedetermined as a predictor for the current pixel P. If the extended line160 passes through the section P2, a weighted average value(3*A+1*B+2)>>2 considering a distance between the neighboring pixels Aand B and a middle point of the section P2 may be determined as apredictor for the current pixel P. If the extended line 160 passesthrough the section P3, a weighted average value (2*A+2*B+2)>>2considering a distance between the neighboring pixels A and B and amiddle point of the section P3 may be determined as a predictor for thecurrent pixel P. If the extended line 160 passes through the section P4,a weighted average value (1*A+3*B+2)>>2 considering a distance betweenthe neighboring pixels A and B and a middle point of the section P4 maybe determined as a predictor for the current pixel P. If the extendedline 160 passes through the section P5, a value of the neighboring pixelB may be determined as a predictor for the current pixel P.

Also, if two neighboring pixels, that is, the neighboring pixel A on theup side and the neighboring pixel B on the left side meet the extendedline 160 as shown in FIG. 16A, an average value of the neighboring pixelA and the neighboring pixel B may be used as a predictor for the currentpixel P, or if (dx*dy) is a positive value, the neighboring pixel A onthe up side may be used, and if (dx*dy) is a negative value, theneighboring pixel B on the left side may be used.

The intra prediction modes having various directionalities shown inTable 1 may be predetermined by an encoding side and a decoding side,and only an index of an intra prediction mode of each coding unit may betransmitted.

FIG. 17 is a reference diagram for explaining a bi-linear mode accordingto an exemplary embodiment.

Referring to FIG. 17, in the bi-linear mode, a geometric average iscalculated by considering a value of a current pixel P 170 in a currentluminance component coding unit, which is to be predicted, values ofpixels on upper, lower, left, and right boundaries of the currentluminance component coding unit, and the distances between the currentpixel P 170 and the upper, lower, left, and right boundaries of thecurrent luminance component coding unit. The geometric average is thenused as a predictor of the current pixel P 170. For example, in thebi-linear mode, a geometric average calculated using a virtual pixel A171, a virtual pixel B 172, a pixel D 176, and a pixel E 177 located tothe upper, lower, left, and right sides of the current pixel P 170, andthe distances between the current pixel P 170 and the upper, lower,left, and right boundaries of the current luminance component codingunit, is used as a predictor of the current pixel P 170. Since thebi-linear mode is one of the intra prediction modes, neighboring pixelsthat have been encoded and restored, and belong to previous luminancecomponent coding units, are used as reference pixels for prediction.Thus, pixel values in the current luminance component coding unit arenot used, but virtual pixel values calculated using neighboring pixelslocated to the upper and left sides of the current luminance componentcoding unit are used as the pixel A 171 and the pixel B 172.

Specifically, first, a value of a virtual pixel C 173 on a lowerrightmost point of the current luminance component coding unit iscalculated by calculating an average of values of a neighboring pixel(right-up pixel) 174 on an upper rightmost point of the currentluminance component coding unit and a neighboring pixel (left-downpixel) 175 on a lower leftmost point of the current luminance componentcoding unit, as expressed in Equation 1 below:

C=0.5(LeftDownPixel+RightUpPixel)  [Equation 1]

Next, a value of the virtual pixel A 171 located on a lowermost boundaryof the current luminance component coding unit when the current pixel P170 is extended downward by considering the distance W1 between thecurrent pixel P 170 and the left boundary of the current luminancecomponent coding unit and the distance W2 between the current pixel P170 and the right boundary of the current luminance component codingunit, is calculated by using Equation 2 below:

A=(C*W1+LeftDownPixel*W2)/(W1+W2);  [Equation 2]

A=(C*W1+LeftDownPixel*W2+((W1+W2)/2))/(W1+W2) When a value of W1+W2 inEquation 2 is a power of 2, like 2̂n,A=(C*W1+LeftDownPixel*W2+((W1+W2)/2))/(W1+W2) may be calculated by shiftoperation as A=(C*W1+LeftDownPixel*W2+2̂(n−1))>>n without division.

Similarly, a value of the virtual pixel B 172 located on a rightmostboundary of the current luminance component coding unit when the currentpixel P 170 is extended in the right direction by considering thedistance h1 between the current pixel P 170 and the upper boundary ofthe current luminance component coding unit and the distance h2 betweenthe current pixel P 170 and the lower boundary of the current luminancecomponent coding unit, is calculated by using Equation 3 below:

B=(C*h1+RightUpPixel*h2)/(h1+h2)

B=(C*h1+RightUpPixel*h2+((h1+h2)/2))/(h1+h2)  [Equation 3]

When a value of h1+h2 in Equation 3 is a power of 2, like 2̂m,B=(C*h1+RightUpPixel*h2+((h1+h2)/2))/(h1+h2) may be calculated by shiftoperation as B=(C*h1+RightUpPixel*h2+2̂(m−1))>>m without division.

Once the values of the virtual pixel B 172 on the right border and thevirtual pixel A 171 on the down border of the current pixel P 170 aredetermined by using Equations 1 through 3, a predictor for the currentpixel P 170 may be determined by using an average value of A+B+D+E. Indetail, a weighted average value considering a distance between thecurrent pixel P 170 and the virtual pixel A 171, the virtual pixel B172, the pixel D 176, and the pixel E 177 or an average value of A+B+D+Emay be used as a predictor for the current pixel P 170. For example, ifa weighted average value is used and the size of block is 16×16, apredictor for the current pixel P may be obtained as(h1*A+h2*D+W1*B+W2*E+16)>>5. Such bilinear prediction is applied to allpixels in the current coding unit, and a prediction coding unit of thecurrent coding unit in a bilinear prediction mode is generated.

According to an exemplary embodiment, prediction encoding is performedaccording to various intra prediction modes determined according to thesize of a luminance component coding unit, thereby allowing efficientvideo compression based on characteristics of an image.

Since a greater number of intra prediction modes than intra predictionmodes used in a conventional codec are used according to a size of acoding unit according to an exemplary embodiment, compatibility with theconventional codec may become a problem. In a conventional art, 9 intraprediction modes at the most may be used as shown in FIGS. 14A and 14B.Accordingly, it is necessary to map intra prediction modes havingvarious directions selected according to an exemplary embodiment to oneof a smaller number of intra prediction modes. That is, when a number ofavailable intra prediction modes of a current coding unit is N1 (N1 isan integer), in order to make the available intra prediction modes ofthe current coding unit compatible with a coding unit of a predeterminedsize including N2 (N2 is an integer different from N1) intra predictionmodes, the intra prediction modes of the current coding unit may bemapped to an intra prediction mode having a most similar direction fromamong the N2 intra prediction modes. For example, a total of 33 intraprediction modes are available as shown in Table 1 in the current codingunit, and it is assumed that an intra prediction mode finally applied tothe current coding unit is the mode 14, that is, (dx,dy)=(4,3), having adirectivity of tan⁻¹(3/4)≈36.87 (degrees). In this case, in order tomatch the intra prediction mode applied to the current block to one of 9intra prediction modes as shown in FIGS. 14A and 14B, the mode 4(down_right) mode having a most similar directivity to the directivityof 36.87 (degrees) may be selected. That is, the mode 14 of Table 1 maybe mapped to the mode 4 shown in FIG. 14B. Likewise, if an intraprediction mode applied to the current coding unit is selected to be themode 15, that is, (dx,dy)=(1,11), from among the 33 available intraprediction modes of Table 1, since a directivity of the intra predictionmode applied to the current coding unit is tan⁻¹(11)≈84.80 (degrees),the mode 0 (vertical) of FIG. 14B having a most similar directivity tothe directivity 84.80 (degrees) may be mapped to the mode 15.

Meanwhile, in order to decode a luminance component coding unit encodedvia intra prediction, prediction mode information is required todetermine which intra prediction mode is used to encode a currentluminance component coding unit. Accordingly, intra prediction modeinformation of the current luminance component encoding unit is added toa bitstream when encoding an image. At this time, an overhead mayincrease, thereby decreasing compression efficiency if intra predictionmode information of each luminance component coding unit is added to thebitstream.

Therefore, according to an exemplary embodiment, instead of transmittingthe intra prediction mode information of the current luminance componentcoding unit, which is determined as a result of encoding the currentluminance component coding unit, only a difference value between anactual value of an intra prediction mode and a prediction value of anintra prediction mode, which is predicted from a neighboring luminancecomponent coding unit, is transmitted.

FIG. 18 is a diagram for explaining a process of generating a predictionvalue of an intra prediction mode of a current luminance componentcoding unit A 180, according to an exemplary embodiment.

Referring to FIG. 18, an intra prediction mode of the current luminancecomponent coding unit A 180 may be predicted from intra prediction modesdetermined in neighboring luminance component coding units. For example,when an intra prediction mode of a left luminance component coding unitB 181 is mode 3, and an intra prediction mode of an upper luminancecomponent coding unit C 182 is mode 4, the intra prediction mode of thecurrent luminance component coding unit A 180 may be predicted to bemode 3, which has a smaller value from among the intra prediction modesof the upper luminance component coding unit C 182 and the leftluminance component coding unit B 181. If an intra prediction modedetermined as a result of actually performing intra prediction encodingon the current luminance component coding unit A 180 is mode 4, only 1,i.e., a difference value with mode 3 constituting the intra predictionmode predicted from the neighboring luminance component coding units, istransmitted as intra prediction mode information. A prediction value ofan intra prediction mode of a current luminance component decoding unitis generated in the same manner during decoding, and a difference valuereceived through a bitstream is added to the prediction value, therebyobtaining intra prediction mode information actually applied to thecurrent luminance component decoding unit. In the above description,only the upper and left neighboring coding units C and B 182 and 181 ofthe current luminance component coding unit A 180 are used, butalternatively, the intra prediction mode of the current luminancecomponent coding unit A may be predicted by using other neighboringluminance component coding units E and D of FIG. 18. An intra predictionmode of a luminance component coding unit may be used to predict anintra prediction mode of a chrominance component coding unit that willbe described later.

Meanwhile, since an intra prediction mode actually performed differsaccording to sizes of luminance component coding units, an intraprediction mode predicted from neighboring luminance component codingunits may not match an intra prediction mode of a current luminancecomponent coding unit. Accordingly, in order to predict the intraprediction mode of the current luminance component coding unit from theneighboring luminance component coding units having different sizes, amapping process for mapping different intra prediction modes of theluminance component coding units is required.

FIGS. 19A and 19B are a reference diagrams for explaining a mappingprocess of intra prediction modes between luminance component codingunits having different sizes, according to an exemplary embodiment.

Referring to FIG. 19A, a current luminance component coding unit A 190has a size of 16×16, a left luminance component coding unit B 191 has asize of 8×8, and an upper luminance component coding unit C 192 has asize of 4×4. Also, as described with reference to FIG. 13, numbers ofintra prediction modes usable in luminance component coding unitsrespectively having sizes of 4×4, 8×8, and 16×16 are respectively 9, 9,and 33. Here, since the intra prediction modes usable in the leftluminance component coding unit B 191 and the upper luminance componentcoding unit C 192 are different from the intra prediction modes usablein the current luminance component coding unit A 190, an intraprediction mode predicted from the left and upper luminance componentcoding units B and C 191 and 192 may not be suitable for use as aprediction value of the intra prediction mode of the current luminancecomponent coding unit A 190. Accordingly in the current exemplaryembodiment, the intra prediction modes of the left and upper luminancecomponent coding units B and C 191 and 192 are respectively changed tofirst and second representative intra prediction modes in the mostsimilar direction from among a predetermined number of representativeintra prediction modes, and one of the first and second representativeintra prediction modes, which has a smaller mode value, is selected as afinal representative intra prediction mode. Then, an intra predictionmode having the most similar direction as the final representative intraprediction mode is selected from among the intra prediction modes usablein the current luminance component coding unit A 190 as an intraprediction mode of the current luminance component coding unit A 190.

Alternatively, referring to FIG. 19B, it is assumed that a currentluminance component coding unit A has a size of 16×16, a left luminancecomponent coding unit B has a size of 32×32, and an up luminancecomponent coding unit C has a size of 8×8. Also, it is assumed thatnumbers of available intra prediction modes of the luminance componentcoding units having the sizes of 8×8, 16×16, and 32×32 are respectively9, 9, and 33. Also, it is assumed that an intra prediction mode of theleft luminance component coding unit B is a mode 4, and an intraprediction mode of the up luminance component coding unit C is a mode31. In this case, since the intra prediction modes of the left luminancecomponent coding unit B and the up luminance component coding unit C arenot compatible with each other, each of the intra prediction modes ofthe left luminance component coding unit B and the up luminancecomponent coding unit C is mapped to one of representative intraprediction modes shown in FIG. 20. Since the mode 31 that is the intraprediction mode of the left luminance component coding unit B has adirectivity of (dx,dy)=(4, −3) as shown in Table 1, a mode 5 having amost similar directivity to tan⁻¹(−3/4) from among the representativeintra prediction modes of FIG. 20 is mapped, and since the intraprediction mode mode 4 of the up luminance component coding unit C hasthe same directivity as that of the mode 4 from among the representativeintra prediction modes of FIG. 20, the mode 4 is mapped.

The mode 4 having a smaller mode value from among the mode 5 that is themapped intra prediction mode of the left luminance component coding unitB and the mode 4 that is the mapped intra prediction mode of the upluminance component coding unit C may be determined to be a predictionvalue of an intra prediction mode of the current luminance componentcoding unit, and only a mode difference value between an actual intraprediction mode and a predicted intra prediction mode of the currentluminance component coding unit may be encoded as prediction modeinformation of the current luminance component coding unit.

FIG. 20 is a reference diagram for explaining a process of mapping anintra prediction mode of a neighboring luminance component coding unitto one of representative intra prediction modes. In FIG. 20, a verticalmode 0, a horizontal mode 1, a DC mode 2, a diagonal-left mode 3, adiagonal-right mode 4, a vertical-right mode 5, a horizontal-down mode6, a vertical-left mode 7, and a horizontal-up mode 8 are shown asrepresentative intra prediction modes. However, the representative intraprediction modes are not limited thereto, and may be set to have variousdirectionalities.

Referring to FIG. 20, a predetermined number of representative intraprediction modes are set, and an intra prediction mode of a neighboringluminance component coding unit is mapped as a representative intraprediction mode having the most similar direction. For example, when anintra prediction mode of an upper luminance component coding unit has adirectionality indicated by MODE_A 200, the intra prediction mode MODE_A200 of the upper luminance component coding unit is mapped to mode 1having the most similar direction from among the predeterminedrepresentative intra prediction modes 1 through 9. Similarly, when anintra prediction mode of a left luminance component coding unit has adirectionality indicated by MODE_B 201, the intra prediction mode MODE_B201 of the left luminance component coding unit is mapped to mode 5having the most similar direction from among the predeterminedrepresentative intra prediction modes 1 through 9.

Then, one of first and second representative intra prediction modeshaving a smaller mode value is selected as a representative intraprediction mode of a final neighboring luminance component coding unit.A representative intra prediction mode having a smaller mode value iselected since a smaller mode value is generally set for intra predictionmodes that occur more frequently. In other words, when different intraprediction modes are predicted based on the neighboring luminancecomponent coding units, an intra prediction mode having a smaller modevalue is more likely to occur. Accordingly, when different intraprediction modes are competing with each other, an intra prediction modehaving a smaller mode value may be selected as a predictor or an intraprediction mode of the current luminance component coding unit.

Even when a representative intra prediction mode is selected based onthe neighboring luminance component coding units, the selectedrepresentative intra prediction mode may not be used as a predictor ofan intra prediction mode of a current luminance component coding unit.If the current luminance component coding unit A 190 has 33 intraprediction modes, and a number of representative intra prediction modesis 9 as described with reference to FIG. 19, an intra prediction mode ofthe current luminance component coding unit A 190, which corresponds tothe representative intra prediction mode, does not exist. In this case,like mapping an intra prediction mode of a neighboring luminancecomponent coding unit to a representative intra prediction mode asdescribed above, an intra prediction mode having the most similardirection as a representative intra prediction mode selected from amongintra prediction modes according to a size of the current luminancecomponent coding unit may be finally selected as a predictor of theintra prediction mode of the current luminance component coding unit.For example, when a representative intra prediction mode finallyselected based on the neighboring luminance component coding units ofFIG. 20 is mode 1, an intra prediction mode having the directionalitymost similar to the directionality of mode 1 is selected from amongintra prediction modes usable according the size of the currentluminance component coding unit as a predictor of the intra predictionmode of the current luminance component coding unit.

Meanwhile, as described with reference to FIGS. 16 A through 16C, if apredictor for the current pixel P is generated by using neighboringpixels on or close to the extended line 160, the extended line 160 hasactually a directivity of tan⁻¹(dy/dx). In order to calculate thedirectivity, since division (dy/dx) is necessary, calculation is madedown to decimal places when hardware or software is used, therebyincreasing the amount of calculation. Accordingly, a process of settingdx and dy is used in order to reduce the amount of calculation when aprediction direction for selecting neighboring pixels to be used asreference pixels about a pixel in a coding unit is set by using dx, anddy parameters in a similar manner to that described with reference toTable 1.

FIG. 25 is a diagram for explaining a relationship between a currentpixel and neighboring pixels located on an extended line having adirectivity of (dy/dx), according to an exemplary embodiment.

Referring to FIG. 25, it is assumed that a location of the current pixelP is P(j,i), and an up neighboring pixel and a left neighboring pixel Blocated on an extended line 2510 having a directivity, that is, agradient, of tan⁻¹(dy/dx) and passing through the current pixel P arerespectively A and B. When it is assumed that locations of upneighboring pixels correspond to an X-axis on a coordinate plane, andlocations of left neighboring pixels correspond to a y-axis on thecoordinate plate, the up neighboring pixel A is located at(j+i*dx/dy,0), and the left neighboring pixel B is located at(0,i+j*dy/dx). Accordingly, in order to determine any one of the upneighboring pixel A and the left neighboring pixel B for predicting thecurrent pixel P, division, such as dx/dy or dy/dx, is required. Suchdivision is very complex as described above, thereby reducing acalculation speed of software or hardware.

Accordingly, a value of any one of dx and dy representing a directivityof a prediction mode for determining neighboring pixels may bedetermined to be a power of 2. That is, when n and m are integers, dxand dy may be 2̂n and 2̂m, respectively.

Referring to FIG. 25, if the left neighboring pixel B is used as apredictor for the current pixel P and dx has a value of 2̂n, j*dy/dxnecessary to determine (0,i+j*dy/dx) that is a location of the leftneighboring pixel B becomes (j*dy/(2̂n)), and division using such a powerof 2 is easily obtained through shift operation as (j*dy)>>n, therebyreducing the amount of calculation.

Likewise, if the up neighboring pixel A is used as a predictor for thecurrent pixel P and dy has a value of 2̂m, i*dx/dy necessary to determine(j+i*dx/dy,0) that is a location of the up neighboring pixel A becomes(i*dx)/(2̂m), and division using such a power of 2 is easily obtainedthrough shift operation as (i*dx)>>m.

FIG. 26 is a diagram for explaining a change in a neighboring pixellocated on an extended line having a directivity of (dx,dy) according toa location of a current pixel, according to an exemplary embodiment.

As a neighboring pixel necessary for prediction according to a locationof a current pixel, any one of an up neighboring pixel and a leftneighboring pixel is selected.

Referring to FIG. 26, when a current pixel 2610 is P(j,i) and ispredicted by using a neighboring pixel located in a predictiondirection, an up pixel A is used to predict the current pixel P 2610.When the current pixel 2610 is Q(b,a), a left pixel B is used to predictthe current pixel Q 2620.

If only a dy component of a y-axis direction from among (dx, dy)representing a prediction direction has a power of 2 like 2̂m, while theup pixel A in FIG. 26 may be determined through shift operation withoutdivision such as (j+(i*dx)>>m, 0), the left pixel B requires divisionsuch as (0, a+b*2̂m/dx). Accordingly, in order to exclude division when apredictor is generated for all pixels of a current block, all of dx anddy may have a type of power of 2.

FIGS. 27 and 28 are diagrams for explaining a method of determining anintra prediction mode direction, according to exemplary embodiments.

In general, there are many cases where linear patterns shown in an imageor a video signal are vertical or horizontal. Accordingly, when intraprediction modes having various directivities are defined by usingparameters dx and dy, image coding efficiency may be improved bydefining values dx and dy as follows.

In detail, if dy has a fixed value of 2̂m, an absolute value of dx may beset so that a distance between prediction directions close to a verticaldirection is narrow, and a distance between prediction modes closer to ahorizontal direction is wider. For example, referring to FIG. 27, if dyhas a value of 2̂4, that is, 16, a value of dx may be set to be 1, 2, 3,4, 6, 9, 12, 16,0, −1, −2, −3, −4, −6, −9, −12, and −16 so that adistance between prediction directions close to a vertical direction isnarrow and a distance between prediction modes closer to a horizontaldirection is wider.

Likewise, if dx has a fixed value of 2̂n, an absolute value of dy may beset so that a distance between prediction directions close to ahorizontal direction is narrow and a distance between prediction modescloser to a vertical direction is wider. For example, referring to FIG.28, if dx has a value of 2̂4, that is, 16, a value of dy may be set to be1, 2, 3, 4, 6, 9, 12, 16,0, −1, −2, −3, −4, −6, −9, −12, and −16 so thata distance between prediction directions close to a horizontal directionis narrow and a distance between prediction modes closer to a verticaldirection is wider.

Also, when one of values of dx and dy is fixed, the remaining value maybe set to be increased according to a prediction mode. For example, ifdy is fixed, a distance between dx may be set to be increased by apredetermined value. Also, an angle of a horizontal direction and avertical direction may be divided in predetermined units, and such anincreased amount may be set in each of the divided angles. For example,if dy is fixed, a value of dx may be set to have an increased amount ofa in a section less than 15 degrees, an increased amount of b in asection between 15 degrees and 30 degrees, and an increased width of cin a section greater than 30 degrees. In this case, in order to havesuch a shape as shown in FIG. 25, the value of dx may be set to satisfya relationship of a<b<c.

For example, prediction modes described with reference to FIGS. 25through 28 may be defined as a prediction mode having a directivity oftan⁻¹(dy/dx) by using (dx, dy) as shown in Tables 2 through 4.

TABLE 2 dx Dy −32 32 −26 32 −21 32 −17 32 −13 32 −9 32 −5 32 −2 32 0 322 32 5 32 9 32 13 32 17 32 21 32 26 32 32 32 32 −26 32 −21 32 −17 32 −1332 −9 32 −5 32 −2 32 0 32 2 32 5 32 9 32 13 32 17 32 21 32 26 32 32

TABLE 3 dx Dy −32 32 −25 32 9 32 −14 32 −10 32 −6 32 −3 32 −1 32 0 32 132 3 32 6 32 10 32 14 32 19 32 25 32 32 32 32 −25 32 −19 32 −14 32 −1032 −6 32 −3 32 −1 32 0 32 1 32 3 32 6 32 10 32 14 32 19 32 25 32 32

TABLE 4 dx Dy −32 32 −27 32 −23 32 −19 32 −15 32 −11 32 −7 32 −3 32 0 323 32 7 32 11 32 15 32 19 32 23 32 27 32 32 32 32 −27 32 −23 32 −19 32−15 32 −11 32 −7 32 −3 32 0 32 3 32 7 32 11 32 15 32 19 32 23 32 27 3232

For example, referring to Table 2, a prediction mode having adirectionality of tan⁻¹(dy/dx) by using (dx, dy) represented as one of(−32, 32), (−26, 32), (−21, 32), (−17, 32), (−13, 32), (−9, 32), (−5,32), (−2, 32), (0.32), (2, 32), (5, 32), (9, 32), (13, 32), (17,32),(21, 32), (26, 32), (32, 32), (32, −26), (32, −21), (32, −17), (32,−13), (32, −9), (32, −5), (32, −2), (32, 0), (32, 2), (32, 5), (32, 9),(32, 13), (32, 17), (32, 21), (32, 26) and (32, 32).

FIG. 21 is a diagram for explaining candidate intra prediction modesapplied to a chrominance component coding unit, according to anexemplary embodiment.

Referring to FIG. 21, candidate intra prediction modes applied whileintra predicting a chrominance component coding unit include a verticalmode, a horizontal mode, a DC mode, a plane mode, and an intraprediction mode finally determined for a luminance component coding unitcorresponding to a current chrominance component coding unit asdescribed above. Also, as described above, a luminance component codingunit and a chrominance component coding unit, which are intra predicted,may be one of image signals having color formats of 4:2:0, 4:2:2, and4:4:4 defined in a YCbCr (or YUV) color domain. An intra prediction modehaving a minimum cost from among a plurality of usable intra predictionmodes is selected as an intra prediction mode of the luminance componentcoding unit, based on cost calculation, such as a R-D cost. Costs of thecandidate intra prediction modes are each calculated, and a candidateintra prediction mode having a minimum cost is selected as a final intraprediction mode of the chrominance component coding unit.

FIG. 22 is a block diagram of an intra prediction apparatus 2200 of animage, according to an exemplary embodiment. The intra predictionapparatus 2200 according to the current embodiment of the presentinvention may operate as an intra predictor 410 of the image encoder 400of FIG. 4, and the intra predictor 550 of the image decoder 500 of FIG.5.

Referring to FIG. 22, the intra prediction apparatus 2200 includes aluminance intra predictor 2210 and a chrominance intra predictor 2220.As described above, the luminance intra predictor 2210 selects candidateintra prediction modes to be applied according to a size of a currentluminance component coding unit, based on a size of each luminancecomponent coding unit split according to a maximum coding unit and amaximum depth, and applies the determined candidate intra predictionmodes to the current luminance component coding unit to perform intraprediction on the current luminance component coding unit. The luminanceintra predictor 2210 determines an optimum intra prediction mode havinga minimum cost as a final intra prediction mode of the current luminancecomponent coding unit based on costs according to an error value betweena prediction coding unit generated via intra prediction, and an originalluminance component coding unit.

The chrominance intra predictor 2220 calculates costs according to avertical mode, a horizontal mode, a DC mode, a plane mode, and the finalintra prediction mode of the luminance component coding unitcorresponding to a current chrominance component coding unit, anddetermines an intra prediction mode having a minimum cost as a finalintra prediction mode of the current chrominance component coding unit.

Meanwhile, when the intra prediction apparatus 2200 of FIG. 22 isapplied to a decoding apparatus, sizes of current luminance andchrominance component decoding units are determined by using a maximumcoding unit and depth information constituting hierarchical splitinformation of the maximum coding unit, which are extracted from abitstream by using the entropy decoder 520 of FIG. 5, and an intraprediction mode to be performed is determined by using information aboutan intra prediction mode information applied to the current luminanceand chrominance component decoding units. Also, the intra predictionapparatus 2200 generates a prediction decoding unit by performing intraprediction on each luminance and chrominance component decoding unitsaccording to the extracted intra prediction mode. The predictiondecoding unit is added to residual data restored from a bitstream, andthus the current luminance and chrominance component decoding units aredecoded.

FIG. 23 is a flowchart illustrating a method of determining an intraprediction mode of a coding unit, according to an exemplary embodiment.

Referring to FIG. 23, a current picture of a luminance component issplit into at least one luminance component coding unit based on amaximum coding unit and a depth constituting hierarchical splitinformation of the maximum coding unit, in operation 2310.

In operation 2320, an intra prediction mode of the luminance componentcoding unit is determined. As described above, the intra prediction modeof the luminance component coding unit is determined by selectingcandidate intra prediction modes to be applied based on a size of theluminance component coding unit, performing intra prediction on theluminance component coding unit by applying the candidate intraprediction modes on the luminance component coding unit, and thendetermining an optimum intra prediction mode having a minimum cost asthe intra prediction mode of the luminance component coding unit.

In operation 2330, candidate intra prediction modes of a chrominancecomponent coding unit, which include the determined intra predictionmode of the luminance component coding unit, are determined. Asdescribed above, the candidate intra prediction modes applied to thechrominance component coding unit include, aside from the determinedintra prediction mode of the luminance component coding unit, a verticalmode, a horizontal mode, a DC mode, and a plane mode.

In operation 2340, costs of the chrominance component coding unitaccording to the determined candidate intra prediction modes arecompared to determine an intra prediction mode having a minimum cost.

FIG. 24 is a flowchart illustrating a method of determining an intraprediction mode of a decoding unit, according to an exemplaryembodiment.

Referring to FIG. 24, a maximum coding unit and a depth constitutinghierarchical spilt information of the maximum coding unit are extractedfrom a bitstream, in operation 2410.

In operation 2420, a current picture to be decoded is split into aluminance component decoding unit and a chrominance component decodingunit, based on the extracted maximum coding unit and depth.

In operation 2430, information about intra prediction modes applied tothe luminance and chrominance component decoding units is extracted fromthe bitstream.

In operation 2440, intra prediction is performed on the luminance andchrominance component decoding units according to the extracted intraprediction modes, thereby decoding the luminance and chrominancecomponent decoding units.

According to the exemplary embodiments, by adding the intra predictionmode of the luminance component coding unit having variousdirectionality as the intra prediction mode of the chrominance componentcoding unit, the prediction efficiency of an image of a chrominancecomponent, and the prediction efficiency of an entire image can beincreased without having to increase a throughput.

The exemplary embodiments may be embodied as computer programs and canbe implemented in general-use digital computers that execute theprograms by using a computer readable recording medium. Examples of thecomputer readable recording medium include magnetic storage media (e.g.,ROM, floppy disks, hard disks, etc.), optical recording media (e.g.,CD-ROMs, or DVDs), and storage media.

The apparatuses of the exemplary embodiments may include a bus coupledto every unit of the apparatus or coder, at least one processor that isconnected to the bus, the processor for executing commands, and a memoryconnected to the bus to store the commands, received messages, andgenerated messages.

While this invention has been particularly shown and described withreference to the exemplary embodiments, it will be understood by thoseof ordinary skill in the art that various changes in form and detailsmay be made therein without departing from the spirit and scope of theinvention as defined by the appended claims. The exemplary embodimentsshould be considered in a descriptive sense only and not for purposes oflimitation. Therefore, the scope of the invention is defined not by thedetailed description of the invention but by the appended claims, andall differences within the scope will be construed as being included inthe present invention.

What is claimed is:
 1. An apparatus for decoding an image, the apparatuscomprising: an entropy decoder that extracts a maximum coding unit thatis a coding unit in which the current picture is encoded having amaximum size, a depth that indicates hierarchical split information ofthe maximum coding unit, and intra prediction mode information thatindicates an intra prediction mode applied to a luminance componentdecoding unit and a chrominance component decoding unit to be decoded,from a bitstream; and an intra prediction performer that performs intraprediction on the luminance component decoding unit and the chrominancecomponent decoding unit to decode the luminance component decoding unitand the chrominance component decoding unit, according to the extractedintra prediction mode. wherein the intra prediction is performed byusing an extension line having an angle of tan⁻¹(dy/dx) based on a pixelin the luminance component coding unit, wherein dx and dy are integers.2. The apparatus of claim 1, wherein the current picture ishierarchically split from a plurality of the maximum coding units,according to information about a maximum size of a coding unit, intocoding units of coded depths according to depths; a coding unit of acurrent depth is one of rectangular data units split from a coding unitof an upper depth; the coding unit of the current depth is split intocoding units of a lower depth, independently from neighboring codingunits; and coding units of a hierarchical structure comprise encodedcoding units among the coding units split from a maximum coding unit. 3.The apparatus of claim 1, wherein (dx,dy) has an at least one valueselected from among the group consisting of (−32, 32), (−26, 32), (−21,32), (−17, 32), (−13, 32), (−9, 32), (−5, 32), (−2, 32), (0.32), (2,32), (5, 32), (9, 32), (13, 32), (17,32), (21, 32), (26, 32), (32, 32),(32, −26), (32, −21), (32, −17), (32, −13), (32, −9), (32, −5), (32,−2), (32, 0), (32, 2), (32, 5), (32, 9), (32, 13), (32, 17), (32, 21),(32, 26) and (32, 32).
 4. The apparatus of claim 1, wherein a candidateintra prediction modes of the chrominance component coding unit comprisea direct current (DC) mode, a horizontal mode, a vertical mode, and aplane mode.
 5. The apparatus of claim 1, wherein an usable intraprediction mode applied to the at least one luminance component codingunit is determined according to a size of the at least one luminancecomponent coding unit.